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ABSTRACT 

A study of recent graduates of two School of 
Education undergraduate programs illustrates institutional research 
design and utilization issues. A survey mailed to 800 alumni from 5 
graduating classes (1987-1991) resulted in 326 responses from those 
with a Teacher Education major and 120 responses from those with a 
Human Development major, and 10 graduates who had both a Teacher 
Education and a Human Development major. Findings are reported for 
variations in expectations from college by academic major, variations 
in perceived professional growth by academic major, evaluation of 
pre-prof ess i onal teaching experience, early professional challenges 
(work-related problems), differences in work-related problems by 
academic major, relationship of perceived professional growth to 
problems in early teaching, and relationship between perceived 
professional growth , pr e-prof es s i onal teaching exper i ence and over a 1 1 
evaluation of undergraduate education. Survey results were presented 
in separate reports to the Dean, faculty, and program chairpersons. 
The role of both researchers and administrators in institutional 
research utilization is emphasized. Critical design issues noted 
included identifying the appropriate population for the survey, 
identifying subpopulat i ons for future analysis, planning to ensure an 
adequate respondent group for analysis, involving the intended 
audiences of the research in the design phase, designing the survey 
instrument to obtain the content and level of evaluative feedback of 
interest to intended audiences, and designing questions and scales 
which measure the constructs addressed in the research (JB) 
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Abstract 

This paper presents the design, implementation strategies, selected analytical techniques and 
results from a recently completed undergraduate alumni survey. Issues addressed are relevant to 
the design of future alumni research. Topics include: the importance of identifying the 
appropriate population and subpopulations for the survey; planning to ensure an adequate 
respondent group for analysis; involving the intended audiences of the research in the design 
phase; designing the survey instrument to obtain the desired information and evaluative 
feedback; and designing reliable measures of the constructs addressed in the research. The paper 
also illustrates how both the researcher and the administrator have critical roles with respect to 
ensuring the utilization of alumni research. 
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Introduction 

The purpose of this paper is to demonstrate how alumni research can be designed and 
utilized effectively to support a range of institutional functions including assessment, program 
review, program planning and applied research. The paper presents the design, implementation 
strategies, selected analytical techniques and results from a recently completed undergraduate 
alumni survey as well as a discussion of principles and strategies potentially useful for the design 
of future alumni research studies. 

The relevance of alumni research to administrative decision making and various 
university functions has been recognized for some time (Pace, 1979). Recently, alumni research 
has assumed increasing importance for its potential contribution to outcomes assessment (Pike, 
1990). As Williford and Moden (1989) observe, a unique feature of alumni surveys, compared 
with surveys of enrolled students, is the capability of documenting students' assessment of the 
quality of their educational experience tempered by their experiences since graduation. Results 
from alumni research also provide a basis for examining the relationship between satisfaction 
with college and acade.nic major (Richardson, 1993), perception of learning during college (Pike, 
1993a), and subsequent work experience (Pike, 1993b). Of particular importance to institutional 
researchers is the realization that in order to serve any of these functions effectively, the alumni 
research study needs to be designed and the information presented in the appropriate for a and at 
the appropriate level of detail for the intended audiences and users of the results (Moden and 
Williford, 1988). 

Design Issues 

The population for this alumni survey included five graduating classes - from 1987 
through 1991 yielding a total of 800 School of Education alumni. These alumni graduated with a 
Teacher Education and/or a Human Development major. Questionnaires were received from 326 
graduates with a Teacher Education major, 120 graduates with a Human Development major and 
10 graduates who had both a Teacher Education and a Human Development major. 
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The overall response rate for this alumni survey was 58.76 percent and the response rates 
for individual academic programs ranged from 52 to 62 percent. Data from this population 
provided a basis for documenting trends in the relationships between post-graduate experience 
and satisfaction with undergraduate education, and for evaluating the relationship between 
changes in graduates' feedback to revisions made in the curriculum during this time period. 

The identification and size of the survey population are critical design issues that affect 
the ability to analyze variations in the data and to make inferences to populations of interest. 
Generally it is advisable to determine, prior to conducting the study, what subgroup analyses will 
be conducted, and to estimate not only the overall response rate but also the response rates for 
individual departments or classes for which inferences might be made. Statistical formulae, 
taking into account levels of statistical significance and power, may also be used to determine 
sufficient sample sizes to support tests of significance and inferences to the intended population. 

Survey Instrument 

The survey instrument in this alumni study was designed to elicit extensive information 
on graduates' perceptions regarding how well their undergraduate education enhanced various 
intellectual and interpersonal skills; how well their college education prepared them for their 
professional life; what challenges they encountered in their early professional careers; and their 
perceptions regarding how undergraduate education might be improved to better prepare future 
graduates to meet these challenges. The survey addressed three major areas: Graduates' 
Evaluation of their Undergraduate Program, Graduates' Post-Graduate Educational and 
Employment Experience, and Selected Background Information. 

The Evaluation of their Undergraduate Program addressed the following types of 
questions: What were graduates' primary expectation from their undergraduate education? How 
challenging were aspects of their undergraduate education - classes and field experiences? How 
satisfied were they with each of the following: practica, supervision, classes, and university 
resources? And, how much did their education contribute to their growth in different areas, such 
as communication, critical thinking, multi-cultural awareness and values awareness? 
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The section on Post-Graduate Employment focused on employment seeking experience, 
types of positions held since graduation, professional challenges encountered and problems that 
might have been better addressed in the curriculum. Background information included residence 
status, entrance status, undergraduate major, yea:* of graduation, post-graduate education and 
annual income; these data provided a basis for developing an undergraduate and a post-graduate 
profile of the respondents. 

Implementation Strategies 

With some modifications, implementation strategies for this survey followed the 
procedures outlined in Dillman's (1978) book, Mail and Telephone Surveys . Data collection 
efforts included a reminder post-card sent one week after the initial mailing and two subsequent 
follow-up mailings that included the complete survey package. A cover letter from the Dean of 
the School of Education was sent w th each of the complete survey mailings. This letter stressed 
the significance of the project to the School, the importance of each respondent's contribution, an 
assurance of confidentiality, an offer of results and a contact for questions. While assuring 
confidentiality, the surveys included an identification number; this was essential for monitoring 
returns and provided a potential means of merging the survey data with information from 
administrative files. 

Utilization Strategies 

Results from this Alumni Survey were initially presented to the Dean in a comprehensive 
report of the overall survey results. The contents of the report included an analysis of alumni 
responses broken down by year of graduation and by academic program. Following the 
submission of the initial administrative report, results were presented, primarily by means of 
graphs, to the faculty. This presentation highlighted strengths and areas in need of improvement 
for the School as a whole with some mention of differences by academic program. Subsequent 
to the faculty presentation, individual survey reports were prepared for each Program 
Chairperson. These reports contained the responses of alumni who graduated from the specific 
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academic program. Program Chairpersons were encouraged to review these data and share them 
with their faculty in ongoing curricula review and revision. 

The various administrative reports previously described focused primarily on graduates' 
identification of program strengths and areas in need of improvement. Subsequent to producing 
these reports, further analyses were conducted to examine relationships between graduates' 
academic major, their evaluation of specific components of the college experience and their 
overall evaluation of their undergraduate education. The statistical results presented in the 
following section are based primarily on results from these analyses conducted subsequent to the 
dissemination of school and department level reports. 

Results 

Results presented in this section are based on different segments of the total population 
selected in accordance with the subject of the analysis. The first two analyses - focusing on 
variations in expectation from college and perceived professional growth through undergraduate 
education - compare Human Development majors with Teacher Education majors. The 
remaining analyses, reflecting the primary emphasis in the survey, focus only on Teacher 
Education majors. These analyses examine the relationship between pre-professional teaching 
experience and overall evaluation of the undergraduate program; differences in early professional 
challenges by academic major; recommendations for curricular revision by academic major; and 
relationships between perceived professional growth and early professional challenges. The 
analyses examine variation among Early Childhood, Elementary, Secondary and Special 
Education majors as well as between the most distinctive group, Secondary Education, and all 
other Teacher Education majors. 

Variations in Expectations from College by Academic Major 

Preliminary analyses, including Chi-square and correlational analyses, were conducted to 
determine if students in the different teacher education programs - Early Childhood, Elementary, 
Secondary, and Special Education - differed significantly with respect to primary expectations 
from their undergraduate education. However, the results revealed no statistically significant 
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differences among students in the specific teacher education programs. Further Chi-square 
analysis was then conducted comparing all Teacher Education majors with Human Development 
majors. Results from this analysis are presented in Figure 1. 

Figure 1 

Differences in Primary Expectation from Undergraduate Education: 
Teacher Education vs. Human Development Majors 
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Chi-square analysis revealed statistically significant differences in the primary 
educational expectations of students majoring in Teacher Education and Human Development. 
As illustrated in Figure 1, the primary expectation of the majority of Teacher Education majors is 
on their careers. Sixty- seven percent of Teacher Education majors reported Career Preparation 
as their primary expectation. In comparison, Human Development majors expressed greater 
variability with respect to their goals. Some 45 percent of Human Development majors cited a 
Good Education while 3 1 percent identified Career Preparation as their primary goal. 

Results from the comparative analysis of these two majors illustrate the value of this type 
of data for understanding the expectations and guiding the intellectual development and career 
planning of students in various academic majors. These results are potentially useful for student 
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advising and curriculum development. A possible implication is teacher education majors need 
to be encouraged to value their college education as a unique time to enhance their own 
intellectual ability and avail themselves of the opportunity to explore the rich resources of 
knowledge in the various arts, science and humanities disciplines. In contrast, human 
development majors, who seem initially less concerned about career preparation, might be 
encouraged to balance their aspirations for intellectual development with early and systematic 
career planning. 

Variations in Perceived Professional Growth by Academic Major 

Based on the premise that a college education is intended to enhance many aspects of an 
individuals life, the alumni survey requested respondents to rate the extent to which their 
undergraduate education contributed to their professional growth on the following dimensions: 
Communication Skills, Critical Thinking, Decision Making, Factual Knowledge, Multi-Cultural 
Awareness, Problem Solving, Self-Understanding and Values Awareness. Comparative analyses 
were conducted to determine if perceived growth differed significantly between Teacher Education 
and Human Development majors. Results from these analyses are presented in Table 1. 

Table 1 

Relationship of Perceived Professional Growth to Academic Major: 
Teacher Education vs. Human Development Majors 
Percent Who Perceived 'Very Much 1 Growth 



Professional 

Growth 

Components 


Teacher 
Education 

(Percent) 


Human 
Development 

(Percent) 


X2 


Values Awareness 


33.7 


49.5 


14.20 *** 


Self -Understanding 


39.7 


57.4 


13.02** 


Critical Thinking 


35.0 


47.8 


7.38* 



*p < .05 



p 5 .01 



p < .001 
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As displayed in Table 1, statistically significant differences were found on three 
dimensions of perceived professional growth: Critical Thinking, Self-Understanding and Values 
Awareness. Approximately 50 percent or more of Human Development majors, compared with 
less than 40 percent of Teacher Education majors perceive that their undergraduate education 
contributed 'Very Much' to their professional growth on these dimensions. 

The substantive issue addressed in this analysis is critical to a comprehensive assessment 
of the effectiveness of a college education for all students. This study presents a seminal idea 
that might be refined and expanded in future studies. Given the results obtained in this research 
and the importance of assessing professional and intellectual growth in general, more work is 
recommended to establish the theoretical foundation and identify the relevant constructs. Once 
the theoretical foundation is established, appropriate questions could then be developed and 
tested empirically to create valid and reliable measurement scales. Design of these measures 
should reflect the general goals of education, the particular mission of an institution and the 
specific goals of the academic department. 

Evaluation of Pre-Professional Teaching Experience: Quantitative Results 

The alumni survey was designed to reflect the importance of field experience in teacher 
education students' undergraduate education. The survey included several items aSKing 
graduates to evaluate the level of challenge and benefit associated with their pre-practicum and 
full-practicum teaching experiences. Chi-square analyses were conducted examining the 
relationship between graduates' evaluation of their pre-professional teaching experiences and 
their overall evaluation of the teacher education program. Results are presented in Table 2. 
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Table 2 



Relationship between Graduates' Satisfaction with Pre-Professional Teaching Experience 
and their Overall Evaluation of the Teacher Education Program 



Evaluative Responses 


Pre-Professional 
Teaching 
Experience 




Very 
Poorly 
to Fair 


Well 


Very 
Well 


Total 


Pre-Pracfcicum 


Dissatisfied 


56.3 


40.6 


3.1 


100 % (N=32) 


r = .41 


Satisfied 


25.8 


45.4 


28.8 


100%(N=163) 


X^= 49.79*** 


Very Satisfied 


7.1 


35.3 


57.6 


100 % (N=85) 


Full-Practicum 


Dissatisfied 


70.0 


20.0 


10.0 


100%(N=10) 


r = .45 


Satisfied 


59.6 


36.5 


3.9 


100 % (N=52) 


X 2 = 67.09*** 


Very Satisfied 


13.6 


43.7 


42.7 


100%(N=213) 



***p<.001 



The data presented in Table 2 document the statistically significant relationships between 
students' pre-professional field experiences and their overall evaluation of their undergraduate 
education. As indicated by the correlations of .41 for the Pre-Practicum experience and .45 for 
the Full-Practicum experience, there is a moderately strong tendency for those who are satisfied 
with their practicum experiences to express a positive evaluation of their undergraduate teacher 
preparation. 

Evaluation of Pre-Professional Teaching Experience: Qualitative Results 

Consistent with the quantitative results, qualitative data also give strong voice to the 
importance of intensive and meaningful practicum experiences as part of the undergraduate 
program in professional schools such as Education. Alumni comments in this survey emphasize 
the importance of having extensive, responsible classroom experience with quality supervision 
during the undergraduate program. As indicated in the following comments, alumni applaud the 
early classroom experience provided through the Pre-Practicum experience. They also 
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recommend increased opportunity for student teachers to assume full classroom leadership in 
their pre- professional teaching experiences. 

The field practicum starting as early as sophomore year was of great assistance 
to me. This gives students an opportunity to experience many types of education 
settings and look at different teaching styles. 

I believe pre-practicum student teaching is one of the most beneficial components 
of teacher preparation. (The university) should be commended for realizing this 
and beginning student teaching in the sophomore year. 

I am very satisfied with my educational experience . . . The professors and course 
work were interesting and challenging. One suggestion about the full-practicum 
... (is to have) student teachers assume full classroom leadership for one or two 
weeks at the end of the semester as this gives one a real sense of what teaching is 
about. 

I feel I would have benefited from more time in a full-practicum situation coupled 
with methods classroom work . . . My full-practicum was extremely beneficial, 
but it was over before I felt fully confident to teach independently. 

I recommend that students be given more teaching experience ... I wish I had two 
full practicum experiences. 

Early Professional Challenges: Work-Related Problems 

A primary purpose of this Alumni Survey was to learn more about graduates' early 
professional experiences; what challenges they encountered; how they dealt with these 
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challenges; how well they think their undergraduate education prepared them for these 
challenges; and what changes they would recommend in the undergraduate curriculum to better 
prepare future graduates. In this regard, alumni of the teacher education programs were 
presented with a list of major work-related problems often mentioned by beginning teachers and 
were asked to indicate the severity of each problem for them during their first teaching 
assignment. Table 3 illustrates the extent to which all teacher education majors experienced 
these problems. As shown, the three top ranking problems were Teaching Students with 
Different Ability Levels, Maintaining Discipline and Mainstreaming with 40, 37 am? 30 percent 
respectively reporting these as moderate or major problems in their early teaching experience. 



Table 3 

Rank Order of Major Work-Related Problems in First Teaching Assignment 
Percent Reporting 'Moderate' or 'Major' Problems 



Problems 


Percent 


Rank 


Teaching Students with Different 
Ability Levels 


40.4 


1 


Maintaining Discipline 


37.5 


2 


Mainstreaming 


29.7 


3 


Evaluating Students' Performance 


28.1 


4 


Evaluating Own Teaching 


28.0 


5 


Relating to Students' Parents 


23.1 


6 


Motivating Students 


20.2 


7 


Planning Class Instruction 


19.2 


8 


Relating to Administrators 


14.2 


9 


Teaching Students with Different 
Socioeconomic Backgrounds 


13.1 


10 


Develop Rapport with Students 


7.6 


11 



Note . The pcrcenls are based on numbers ranging between 269 and 289. 
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Differences in Work-Related Problems by Academic Major 

Further Chi-square analyses were also conducted to determine if there were significant 
differences in the problems experienced by graduates of different teacher education programs. 
Results revealed most notable differences between Secondary Education and other Teacher 
Education majors. 

As shown in Table 4, a significantly higher percent of Secondary Education, compared 
with other Teacher Education majors, cited Motivating Students, Evaluating their Own Teaching, 
and Teaching Students with Different Socioeconomic Backgrounds as moderate or major 
problems in their first teaching assignment. Although the results are not statistically significant, 
a comparatively higher percent of Secondary Education majors also cited Developing Rapport 
with Students, Relating to Administrators, Maintaining Discipline and Teaching Students with 
Different Ability Levels as major or moderate problems. In contrast, a substantially lower 
percent of Secondary Teachers cited Evaluating Student Performance as a moderate or major 
problem in their first teaching assignment. 

Table 4 

Relationship of New Teachers' Work-Related Problems to their Academic Major 

Secondary Education vs. Other Teacher Education Majors 

Percent Reporting 'Moderate' or 'Major 1 Problem 

Academic Major 

Secondary Other Teacher 
Problems Education Education 
(Percent) (Percent) 

Motivating Students 33.3 19.7 10.51** 

Evaluating Own Teaching 47.4 28.4 8.25 * 

Teaching Students with Different 

Socioeconomic Backgrounds 29.4 1 1.9 7.83 * 

* p<.0S "p<.01 

Note . Chi -square is based on the complete distribution of responses. 
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These results confirm the importance of providing Secondary Education majors with 
intensive preparation for relating to and managing students. The findings indicate that 
introducing strategies to motivate students, maintain discipline and develop rapport with students 
should form the primary core of Secondary Education methods courses. 

The inquiry and analysis presented here are potentially relevant to other academic majors, 
particularly those in professional schools. The topic of early professional challenges or work- 
related problems is relevant to alumni of most academic programs and institutions. Further, 
feedback from the alumni regarding the challenges they confront and the adequacy of their 
preparation should be elicited on a systematic and continuing basis in order to ensure that the 
education offered is relevant and responsive to the needs of the institution's graduates and of the 
professions being served by these graduates. 

Initially, focus groups might be used with a select number of alumni to identify early 
career-related problems and professional challenges. A review of the literature, combined with 
consultation with employers of the alumni and faculty, might expand the initial list. Following 
this exploratory phase, an instrument might be pilot tested. Factor analysis and reliability 
analysis might then be employed in analyzing the data to produce valid and reliable measures of 
early professional problems or challenges. 

Relationship of Undergraduate Major to Early Professional Challenges 

Following the identification of work-related problems experienced in their early career, 
alumni were then asked to indicate which problems ihey think should have been better addressed 
in their undergraduate program. Graduates as a whole most frequently mentioned Teaching 
Students of Different Ability Levels, followed by Maintaining Discipline and Relating to 
Parents. Comparative analysis revealed that graduates from different teacher education majors 
varied to some extent in the ranking of these problems. Special Education majors were quite 
different from the group as a whole; they most frequently mentioned Relating to Parents, 
followed by Teaching Students of Different Ability Levels and Relating to Administrators as 
problems in need of greater attention in the curriculum. Secondary Education majors most 
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frequently cited Maintaining Discipline followed by Teaching Students of Different Ability 
Levels and Motivating Students. The type of information illustrated with these data is potentially 
very helpful to chairpersons, program coordinators and faculty committees in identifying the 
areas that need to be strengthened in the total curriculum and in individual programs. 
Relationship of Perceived Professional Growth to Problems in Early Teaching 

Statistical analyses were conducted to test the hypothesis that perceived professional 
growth during undergraduate education would correlate negatively with the extent to which 
graduates experienced certain problems in their early professional careers. Preliminary analyses 
were based on individual items and focused on the three problems most frequently cited as 
moderate or major problems by the graduates - Teaching Students with Different Ability Levels, 
Maintaining Discipline and Mainstreaming. Perceived professional growth included the 
following dimensions: Communication Skills, Critical Thinking, Decision Making, Factual 
Knowledge, Multi-CuUural Awareness, Problem Solving, Self-Understanding and Values 
Awareness. 

Statistically significant correlation coefficients were found between six of the eight 
perceived professional growth components and two of the top ranking problems: Teaching 
Students of Different Ability Levels and Mainstreaming. The coefficients ranged between -.12 
and -.25 and were significant at the .Ql probability level. Also, perceived growth in Problem 
Solving related significantly, at the .01 probability level, to each of the top three problems: 
Teaching Students with Different Ability Levels ( -.25 ), Mainstreaming ( -.22), and Maintaining 
Discipline ( -.14 ). As indicrted by the negative sign, the more a graduate perceived growth on 
these dimensions, the less likely he or she would have difficulty with these issues. Although the 
magnitude of the relationships found was relatively weak, this preliminary analysis was based 
only on single items. 

Further analysis involved an effort to form more reliable measures with the existing data. 
The following scales were created: Pre-Professional Teaching Experience, Perceived 
Professional Growth, Course Satisfaction, and Teacher-Student Relationship Problems. The 
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development of the scales involved conducting factor analyses to simplify the data, reveal the 
underlying constructs and identify correlated items that might be combined to form these scales. 
The items and response options for each of the scales are summarized in the following outline. 



Pre - Professional Teaching Experience 

How challenging were 

your Pre-Practicum Field experiences and 
your Full-Practicum Field experiences? 

(5-point scale-'Not at AH' to 'Extremely') 

How satisfied were you with the fc 'owing as 
opportunities to relate theory to practice: 

your Pre-Practicum, your Full-Practicum, 

and your Student Teaching? 

(4-point scale -'Very Dissatisfied' to 'Very 
Satisfied') 


Perceived Professional Growth 

To what extent do you feel that your 
undergraduate program of studies and related 
experiences contributed to your professional 
growth and development in each of the 
following areas: 

Communication Skills, Critical Thinking, 
Decision Making, Problem Solving, Self- 
Understanding, and Values Awareness ? 

(5-point scale -'Not at All' to 'Very Much') 


Course Satisfaction 

How satisfied are you with your 

Overall School of Education Courses and 
Methods Courses? 

(4-point scale -'Very Dissatisfied' to 'Very 
Satisfied') 


Teacher-Student Relationship Problems 

Please indicate the severity of the following 

problems for you during your first teaching 

« 

assignment: 

Developing a Rapport with Students, 
Maintaining Discipline, and Motivating 
Students. 

(5-point scale-'Not at All' to 'Major') 
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Table 5 presents the components and reliability levels for each of the scales used in 
further analyses. 

Table 5 



Teacher Education and Experience Scales 



Scale Name 


No. of Items 


Mean 


Standard 
Deviation 


Reliability 
Coefficient 


Pre-Professional 
Teaching Experience 


5 


3.52 


.368 


.76 


Perceived Professional 
Growth 


6 


4.00 


.117 


.86 


Course Satisfaction 


2 


3.24 


.050 


.75 


Teacher-Student 
Relationship Problems 


3 


2.05 


.564 


.70 



Relationship between Perceived Professional Growth, Pre-Professional Teaching 
Experience and Overall Evaluation of Undergraduate Education 

Correlational analyses revealed statistically significant positive relationships between 
Perceived Professional Growth through undergraduate education, an evaluation of Pre- 
Professionai Teaching Experience and the overall Evaluation }f Undergraduate Education. The 
correlation coefficients, significant at the .01 probability level, are .37 for Perceived Professional 
Growth and .44 for Pre-Professional Teaching Experience. Regression analyses were then 
conducted to identify which variables best predicted students' overall evaluation of their 
undergraduate program. The population for the regression was limited to graduates of teacher 
education programs who had complete data for the independent variables and who responded to 
the question which served as the dependent variable, "In your opinion, how well did your 
teacher preparation program prepare you for your present employment?" Regression results are 
presented in Table 6. 
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Table 6 



Stepwise Regression Results Predicting Overall Evaluation of the Teacher 



Preparation Program from Course Satisfaction, 



Pre-Professional Teaching Experience and Perceived Professional Growth 



Independent 
Variables 




Course Satisfaction .55 



.55 



.30 



.30 



.55 



120.08*** 



Pre-Professional 

Teaching .44 .61 .38 .08 .29 83.27*** 

Experience 

Perceived 

Professional .37 .63 .40 .02 .15 59.91*** 

Growth 

*** p<.00l 

As shown in Table 6, three variables contributed significantly to predicting students' 
overall evaluation of their teacher preparation program: Course Satisfaction, Pre-Professional 
Teaching Experience, and Perceived Professional Growth through undergraduate education. 
Among these three variables, Course Satisfaction is the strongest predictor explaining 30 percent 
of the variance in overall satisfaction. With this variable in the equation, Pre-Professional 
Teaching Experience explains an additional 8 percent and Perceived Professional Growth 
another 2 percent. The relative influence of each of the independent variables is also reflected in 
the Beta coefficients which are .55, .29 and .15 respectively for Course Satisfaction, Pre- 
Professional Teaching Experience, and Perceived Professional Growth. Each Beta coefficient 
indicates the amount of change in tb^ dependent variable for every unit change in the 
independent variable. 



ERIC 



20 



' 19 

Summary 

This paper illustrates how both the researcher and the administrator have critical roles 
with respect to ensuring the utilization of institutional research studies in general and alumni 
research in particular. The paper presents design principles, analytical techniques, 
implementation procedures and utilization strategies intended to enhance the contribution of 
alumni research to curriculum review and planning, administrative decision making, outcomes 
assessment and accreditation. Alumni studies enable institutions to meet typical accreditation 
standards requiring follow-up studies of graduates. 

Some of the critical design issues addressed in the discussion include: the importance of 
identifying the appropriate population for the survey; identifying subpopulations for future 
analyses; planning to ensure an adequate respondent group for analysis; involving the intended 
audiences of the research in the design phase; designing the survey instrument to obtain the 
content and level of evaluative feedback of interest to intended audiences; and designing 
questions and scales that will serve as reliable measures of the constructs addressed in the 
research. 

Three critical principles are identified to guide the implementation of the research. The 
research should be conducted in accordance with the highest professional standards. 
Communication with the respondents should be professional, personal and genuinely 
appreciative of their contribution. Well planned and systematic follow-up procedures should be 
implemented to ensure a high response rate. 

The analyses of alumni research data should be planned during the design phase so that 
the questions will be designed and the data obtained at the appropriate level for the analysis. 
Some general guidelines in this area include: preparing customized reports for audiences at 
different levels - deans, chairpersons, program coordinators and faculty; including both 
quantitative and qualitative data in the survey; and ensuring the quantitative data will be at the 
appropriate level for analysis. Methods of analysis will have a direct impact on utilization of the 
results. 
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How the results are presented also affects utilization. With the alumni study reported in 
this paper, the researcher chose to emphasize the positive findings with a deliberate focus on 
program strengths. Areas in need of improvement were presented as constructive possibilities 
for program enhancements. This positive approach, which showed a sensitivity to the faculty 
whose work was being evaluated in this survey, contributed to a positive reception of the 
findings. In addition, the use of graphs also enhanced the accessibility of the findings. 

One of the most important aspects of the substantive findings in this research is the 
valuable feedback graduates provided with respect to challenges they encountered early in their 
professional careers and, more importantly, what recommendations they offered to better prepare 
future graduates to meet these challenges. This research demonstrates that the perspective gained 
through experience is indeed one of the unique values of alumni research (Williford and Moden, 
1989). 

The focus on early professional challenges and the relationship to previous education are 
potentially relevant to other academic majors, particularly those in professional schools. As 
noted previously, feedback from the alumni regarding the challenges they confront and the 
adequacy of their preparation should be elicited on a systematic and continuing basis in order to 
ensure that the education offered is relevant and responsive to the needs of the institution's 
graduates and of the professions being served by these graduates. 

Finally, the high level of administrative support provided the essential context for the 
successful utilization of this alumni research study. Administrative vision and leadership, 
combined with a strong commitment to planning based on research, are essential determinants of 
utilization of program research. The alumni survey discussed in this paper benefited from the 
presence of this spirit and commitment with the support of the Dean of the School cf Education 
and, with her leadership, the support of the Chairpersons and Faculty. 
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